Model selection, large deviations and consistency of data-driven tests

We consider three general classes of data-driven statistical tests. Neyman's smooth tests, data-driven score tests and data-driven score tests for statistical inverse problems serve as important special examples for the classes of tests under consideration. Our tests are additionally incorporated with model selection rules. The rules are based on the penalization idea. Most of the optimal penalties, derived in statistical literature, can be used in our tests. We prove general consistency theorems for the tests from those classes. Our proofs make use of large deviations inequalities for deterministic and random quadratic forms. The paper shows that the tests can be applied for simple and composite parametric, semi- and nonparametric hypotheses. Applications to testing in statistical inverse problems and statistics for stochastic processes are also presented

Robust nonparametric detection of objects in noisy images

We propose a novel statistical hypothesis testing method for detection of objects in noisy images. The method uses results from percolation theory and random graph theory. We present an algorithm that allows to detect objects of unknown shapes in the presence of nonparametric noise of unknown level and of unknown distribution. No boundary shape constraints are imposed on the object, only a weak bulk condition for the object's interior is required. The algorithm has linear complexity and exponential accuracy and is appropriate for real-time systems. In this paper, we develop further the mathematical formalism of our method and explore important connections to the mathematical theory of percolation and statistical physics. We prove results on consistency and algorithmic complexity of our testing procedure. In addition, we address not only an asymptotic behavior of the method, but also a finite sample performance of our test

Robust nonparametric detection of objects in noisy images

We propose a novel statistical hypothesis testing method for detection of objects in noisy images. The method uses results from percolation theory and random graph theory. We present an algorithm that allows to detect objects of unknown shapes in the presence of nonparametric noise of unknown level and of unknown distribution. No boundary shape constraints are imposed on the object, only a weak bulk condition for the object's interior is required. The algorithm has linear complexity and exponential accuracy and is appropriate for real-time systems. In this paper, we develop further the mathematical formalism of our method and explore important connections to the mathematical theory of percolation and statistical physics. We prove results on consistency and algorithmic complexity of our testing procedure. In addition, we address not only an asymptotic behavior of the method, but also a finite sample performance of our test.Comment: This paper initially appeared in 2010 as EURANDOM Report 2010-049. Link to the abstract at EURANDOM repository: http://www.eurandom.tue.nl/reports/2010/049-abstract.pdf Link to the paper at EURANDOM repository: http://www.eurandom.tue.nl/reports/2010/049-report.pd

Data-driven efficient score tests for deconvolution problems

We consider testing statistical hypotheses about densities of signals in deconvolution models. A new approach to this problem is proposed. We constructed score tests for the deconvolution with the known noise density and efficient score tests for the case of unknown density. The tests are incorporated with model selection rules to choose reasonable model dimensions automatically by the data. Consistency of the tests is proved

Model selection, large deviations and consistency of data-driven tests

We consider three general classes of data-driven statistical tests. Neyman's smooth tests, data-driven score tests and data-driven score tests for statistical inverse problems serve as important special examples for the classes of tests under consideration. Our tests are additionally incorporated with model selection rules. The rules are based on the penalization idea. Most of the optimal penalties, derived in statistical literature, can be used in our tests. We prove general consistency theorems for the tests from those classes. Our proofs make use of large deviations inequalities for deterministic and random quadratic forms. The paper shows that the tests can be applied for simple and composite parametric, semi- and nonparametric hypotheses. Applications to testing in statistical inverse problems and statistics for stochastic processes are also presented.

Algebraic polynomials and moments of stochastic integrals

Introduction. Connections between special algebraic polynomials and stochastic integrals have a long history (see Wiener [1938], Ito [1951]), and received considerable attention in stochastic analysis (Ikeda and Watanabe [1989], Carlen and Kree [1991], Borodin and Salminen [2002]). Fruitful applications of special polynomials have been found in the theory of Markov processes (Kendall [1959], Karlin and Mc-Gregor [1957]), financial mathematics (Schoutens [2000]), statistics (Diaconis and Zabell [1991]). The book Schoutens [2000] contains an extensive overview of this field of stochastic analysis and its applications. In this paper, we study a different type of applications of polynomials to stochastic integration. We show that not only properties of special systems of orthogonal polynomials can be used in stochastic analysis, but in fact that elementary properties of many general classes of polynomials lead to fruitful applications in stochastics

Data-driven goodness-of-fit tests

No abstract

Detection of objects in noisy images and site percolation on square lattices

We propose a novel probabilistic method for detection of objects in noisy images. The method uses results from percolation and random graph theories. We present an algorithm that allows to detect objects of unknown shapes in the presence of random noise. Our procedure substantially differs from wavelets-based algorithms. The algorithm has linear complexity and exponential accuracy and is appropriate for real-time systems. We prove results on consistency and algorithmic complexity of our procedure